Moving imager camera for track and range capture

ABSTRACT

A precision motion platform carrying an imaging device under a large-field-coverage lens enables capture of high resolution imagery over the full field in an instantaneous telephoto mode and wide-angle coverage through temporal integration. The device permits automated tracking and scanning without movement of a camera body or lens. Coupled use of two or more devices enables automated range computation without the need for subsequent epipolar rectification. The imager motion enables sample integration for resolution enhancement. The control methods for imager positioning enable decreasing the blur caused by both the motion of the moving imager or the motion of an object&#39;s image that the imager is intended to capture.

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of Baker et al.'s copendingU.S. Provisional Patent Application No. 60/302,761, entitled “MOVINGIMAGER CAMERA FOR TRACKING, SCANNING, RANGE AND SUPER-RESOLUTION,” filedDec. 11, 1996, which is incorporated herein by reference in itsentirety. This application is related to Woodfill et al.'s copendingU.S. patent application Ser. No. 08/839,767, filed Apr. 28, 1997,entitled “Data Processing System and Method,” which is incorporatedherein by reference in its entirety.

TECHNICAL FIELD

[0002] This invention relates generally to computer input devices, andmore particularly to digital image capture devices used to provideranging and tracking information for a computer system.

BACKGROUND ART

[0003] The range of an object, i.e. the distance to the object from anobservation site, can be determined by the analysis of two or morespatially separated images (often referred to as “binocular images” whenthere are two images) that are taken from the observation site. In rangecomputation from simultaneously acquired binocular digital images, thearea of processing is limited to the visible region of overlap betweenthe two images. To maintain a reasonable region of overlap usuallynecessitates redirecting the optical axes of the cameras (i.e. changingtheir vergence) which introduces other problems including a usualnecessity to resample the imagery. As is well known to those skilled inthe art, “vergence” means the angle between the optical axes of thelenses.

[0004] The processing of binocular or multi-view imagery for rangecomputation is easiest when the optical axes are parallel and theimaging surfaces are coplanar—in what is termed parallel epipolargeometry. Because verging the optical axes to optimize the region ofimage overlap eliminates image-surface co-planarity, the complexities ofcalculating range increases significantly with nonparallel viewing. Thisis further compounded when viewing objects at a variety of azimuths anddistances where adjustments in the view direction as well as vergingwould be necessary to retain sufficient image overlap.

[0005] For computational purposes, the frame of reference for scenedescription is usually tied to image location, so changing the imagelocation through vergence adjustments necessitates reconfiguring theframe of reference. Again, adjusting a system's frame of referenceincreases the computational and conceptual complexity of its analysis.

[0006] A similar situation arises for typical monocular (i.e. singleimage) computer analysis of tracking and scanning in some space beforethe camera. With subjects able to operate over a broad region before thecamera, continued observation generally involves use of eitherwide-angle optics or a panning/tilting mechanism to properly direct thecamera's view direction. These control mechanisms are relativelycomplex, must move fairly large pieces of equipment (cameras and theirlenses), and alter the underlying geometric frame of reference for sceneanalysis by rotating the frame of reference with the cameras. Inaddition, the use of wide angle optics works against high resolutionanalysis, as only larger scene detail is visible.

[0007] One approach to solve these acquisition problems in image-basedrange and tracking computation would be to employ greatly oversizedimagers (e.g. imagers having about 3K by 3K or 9×10⁶ elements), andselect corresponding standard-sized windows within these for processing.However, such an approach would be prohibitively expensive. For example,a 1K by 1K imager sells for well over a thousand dollars. Higherresolution imagers are available at considerably greater price.

[0008] A prior art solution to the apparent dichotomy between simpleprocessing (with parallel epipolar geometry) and broad depth andtracking coverage exists in adaptation of perspective-correcting lenssystems as used in “view-camera” or “technical-camera” designs. In suchdesigns, an oversized lens is used to image the scene, and lateralrepositioning of the lens or imaging platform can be used to redirectthe camera without rotating the imaging surface. For single camera usethis enables maintaining lines parallel in the world parallel on theimage plane; in ranging camera use it enables-parallel epipolargeometry.

[0009] For example, in U.S. Pat. Nos. 5,063,441 and 5,142,357 of Liptonet al., devices for use in 3D videography are disclosed. Moreparticularly, Lipton et al. teach devices for capturing binocular imagesfor use in stereo videography (stereo movies), with reduced viewingeyestrain, by using dual and triple camera systems. Briefly stated,Lipton et al teaches an imager controller for epipolar stereo capture invideography, including stereo lenses mounted fixedly together in asingle housing. Stereographics Inc., of San Raphael, Calif., produces aproduct embodying elements of the Lipton et al. patents.

[0010] In the matter of two-dimensional imager control, U.S. Pat. No.5,049,988, of Sefton et al. teaches a system that provides the displayof a video capture window for surveillance applications. Phillips, inU.S. Pat. No. 4,740,839, teaches a TV surveillance system operated bysub-sampling a conventional camera, with a result that resembles theLipton et al. approach of image capture.

[0011] As will be appreciated, image capturing of the prior art usesplanar sensors due to the high cost, lack of availability, andcomplexities involved with the use and manufacture of curved or“spherical” sensors. However, spherical sensors have a number ofadvantages with respect to field of view, view direction, and use instereo image capture that designers of prior art digital imaging camerashave apparently failed to consider.

DISCLOSURE OF THE INVENTION

[0012] The present invention includes a multi-image camera system forautomated stereo ranging applications that provides high-resolutionbroad field coverage over a wide range of subject distances whileenabling retention of the epipolar constraints necessary for efficientstereo matching. At the same time, the multi-image camera of the presentinvention supports monocular image applications such as object tracking.

[0013] In one embodiment of the present invention, an imaging device ispreferably placed on a three-degree-of-freedom linear motion positioningmechanism whose position can be controlled by a computer. Broad fieldcoverage is preferably attained through use of a wide-anglelarge-coverage lens (e.g. a lens designed for a 35 mm camera). Highresolution is attained through placing the imaging device under thelarge-coverage lens, so the imaging device's immediate field of view isconsiderably narrower than that provided by the lens.

[0014] In contrast with traditional view-camera usage, the presentinvention teaches moving the imaging surface instead of the lens. Thisallows the apparatus to retain co-planarity of the images and, so longas the displacements can be determined to sub-pixel accuracy, maintainsa stable frame of reference for the analysis, all while providing therequired view re-directions. Moving the lens alters the projectiverelationships among observations, whereas moving the imager does not.

[0015] Image focus may be attained through the traditionalrotating-travel focus adjustment although, for quantitativecomputational tasks, this is often unsatisfactory as the center ofprojection may vary with lens rotation. For this situation, ourpreferred embodiment of the moving imager provides a back-focuscapability, as will be described below.

[0016] Computer control of the platform enables positioning accuracy inthe micron range. Stepper motors, for example, operating with full steppositioning, can position a platform along an axis to within a fewmicrons of the desired position. Differential stepper control enablesincreasing the accuracy of this placement (the number of locations wherethis precision is attainable) by one to two orders of magnitude. Finespecification of absolute imager location can also be attained throughuse of interferometers or sonar-based ranging systems, as will bedescribed below in more detail.

[0017] The planar imaging embodiment of the present invention enjoys anumber of advantages over related devices of the prior art. In thepresent invention, an oversized lens is used to provide a wide field ofview of the scene. Movement of the imaging surface under the lensprovides high-resolution view selection, which is the equivalent of viewredirection of pan and tilt motions without the need for a complexmechanism for accomplishing such motions. With accurate positioningknowledge, this form of imaging provides a stable frame of reference forscene computation. Accurate positioning information may be attainedusing the positioning and control systems described below.

[0018] Furthermore, the apparatus of the present invention provides aneconomical solution to the aforementioned problems of the prior art. Forexample, the present invention is operable with relatively economicalhigh-precision imagers that use displacement for direction selection intracking, scanning, and range computation. More particularly, certainembodiments of the present invention place an imaging device on athree-degree-of-freedom linear motion platform whose position can becontrolled by a computer to efficiently and economically provide thedesired direction selection. The third degree of freedom providesback-plane focus control. Back-plane focusing has an advantage fortraditional lens designs where focus from element rotationaldisplacements causes image centering variations.

[0019] In a spherical camera embodiment of the present invention, imagecapture is effected through spherical focal-plane imaging using, forexample, a spherical-faced fiber-optic faceplate. One lens suitable forthis embodiment, the Baker Ball lens, is described in a wartime reportreferenced below. In this spherical-imaging embodiment of the presentinvention, a mechanism is provided that shares many of the designconsiderations of the above-described linear moving-imager camera.Preferably, this embodiment uses a high resolution imaging device (1K by1K elements with 12 bits of resolution at each pixel), mounted behind afiber-optic faceplate that transfers the focal-plane light pattern tothe sensor elements of the imager. Notable in this embodiment of themoving imager is that the lens system has no particular optical axis oraxis along which any imaging surfaces are preferentially oriented.

[0020] In addition, the spherical camera embodiment of the presentinvention has a number of advantages over prior art planar imagingcameras. For example, the spherical camera embodiment we describe has 1)excellent optical resolution attainable since the focal surface does notneed to be made planar (the lens is diffraction limited); 2) greatersimplicity due to the advantages of rotational over lineardisplacements; 3) a greatly increased undistorted field of view; 4)exponentially less lens illumination fall-off with eccentricity (cosineof radial displacement rather than the fourth power of the cosine); 5)greater effective pan and tilt velocities; and 6) opportunity to study(via simulation) aspects of human psychophysical behavior in a mechanismof similar geometry.

[0021] Certain embodiments of the present invention, both of the linearand spherical lens, provide lenses in multiple housings to increase theflexibility, reliability, and range finding quality of the system. Amonocular version of the present invention requires only a single lens,and multi-lens versions of the present invention can use two or morelenses for multi-image ranging analysis.

[0022] These and other advantages of the present invention will becomeapparent upon reading the following detailed descriptions and studyingthe various figures of the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023]FIG. 1 is a diagrammatic view of a monocular imager in accordancewith the present invention;

[0024]FIG. 2 is a diagrammatic view of a multi-camera (binocular in thiscase) imager in accordance with the present invention;

[0025]FIG. 3 is an illustration of an imager under a wide angle lens ofthe present invention;

[0026]FIGS. 4 and 5 illustrate the use of a binocular imager inadjusting the range of available stereoscopic images in accordance withthe present invention;

[0027]FIGS. 6 and 7 illustrate an alignment mechanism in accordance withone embodiment of the present invention;

[0028]FIGS. 8 and 9 provide diagrammatic views of a multi-camera imagerwherein each moving imager utilizes a three platform positioningmechanism; and

[0029]FIG. 10 is an illustration of a spherical imager of the presentinvention.

BEST MODES FOR CARRYING OUT THE INVENTION

[0030] With reference to FIG. 1, a monocular imager apparatus or movingimager “camera” 10 in accordance with one embodiment of the presentinvention will now be described. The moving imager camera 10 includes alens 12, and an imaging array, imaging device, or “imager” 14. Asindicated by the arrows 16, 17, 18, 19, 100, and 102, the imager iscapable of independent motion along the three degrees of freedom definedby an X-Y-Z axis. This means that the imager 14 can move both in-planeand out-of-plane with respect to the lens 12 for view selection andfocus, and may also be synchronized for rectangular displacements 20 inthe imager plane P to provide super-resolution frame integration. Thedisplacements 20 are preferably rectangular, although they arerepresented by a circle in this figure. As will be appreciated by thoseskilled in the art, “rectangular” means any appropriate movement of theimager in the plane P to provide the necessary local sampling. Forexample, these rectangular type displacements are described in Tagami etal.'s U.S. Pat. No. 5,402,171, issued Mar. 28, 1995.

[0031] Preferably the motions of the moving imager camera 10 arecontrolled by a computer controller 21 through the use of suitabletransducing mechanisms. While high precision encoders or the like may beworkable, it is preferable that an extremely high precision measurementsystem such as one utilizing an interferometer or sonar based mechanismbe used. For imparting motion, an actuator such as a stepper motor issuitable. A variety of commercially available actuators are suitable forimparting motion along the X, Y and Z axes. For example, a steppermotor, along with appropriate gearing and linkages, available fromPhysik Instruments of Germany (with suppliers in the U.S.) has beenfound to be suitable for imparting these motions. Those skilled in theart will be familiar with the design and implementation of the variousactuators, high precision measurement systems, and the necessarycomputer control. A few illustrative embodiments of actuators andmeasurement systems suitable for controlling and determining theposition of the imager 14 will be described in more detail below.

[0032] The lens 12 can be a relatively inexpensive, wide angle lens,commercially available from a variety of sources. For example, a 35millimeter (“mm”) camera lens works well for the present invention. Onesuitable imager 14 is an imaging array having a resolution of at least512×512 pixels, although imaging arrays of lesser resolution will work.Such imaging arrays are available from a variety of sources, includingSharp Electronics, Inc. of Japan (with suppliers in the U.S.). Anothercontemplated imager 14 is a photosensitive integrated circuit having anarray of photosensitive elements. One advantage in this particularembodiment is that significant processing of the signals generated atthe photosensitive elements could be performed locally within thephotosensitive integrated circuit.

[0033] In FIG. 2, a multi-camera or “binocular” version of the presentinvention is illustrated. More particularly, an imager apparatus or“camera” 22 includes a pair of independent monocular apparatus 10 a and10 b (see FIG. 1 for corresponding elements) that are operated inconcert. However, it should be noted that any number n of monocularapparatus 10 can be used, in concert, for particular applications. Forexample, n=3, 4, 5, or 6 are useful for certain applications.

[0034] The apparatus 10 a includes a lens 12 a, and an imaging array or“imager” 14 a. As noted by the arrows 16 a, 17 a, 18 a, and 19 a, theimager 14 a is capable of independent in-plane motion with respect tothe lens 12 a for view selection. In certain embodiments, the imager 10a may also move in the out-of-plane direction for focus control. Inaddition, the imager 14 a is preferably synchronized for rectangulardisplacements (as described above with reference to FIG. 1) 20 a in theplane Pa of the imager to provide super-resolution frame integration.The various components of the apparatus 10 a (e.g. the lens, motionactuators, imager, etc.) are as described previously with respect to theapparatus 10 of FIG. 1.

[0035] Similarly, the apparatus 10 b includes a lens 12 b, and animaging array or “imager” 14 b. As noted by the arrows 16 b, 17 b, 18 b,and 19 b, the imager 14 b is capable of independent in-plane motion withrespect to the lens 12 b for view selection. In certain embodiments, theapparatus 10 b may also move in the out-of-plane direction for focuscontrol. In addition, the imager 14 b is preferably synchronized forrectangular displacements (as defined above) 20 b in the plane Pb of theimager to provide super-resolution frame integration. The variouscomponents of the apparatus 10 b (e.g. the lens, motion actuators,imager, etc.) are as described previously with respect to the apparatus10 of FIG. 1.

[0036] The apparatus 10 a and 10 b of camera 22 operate as follows. Thelateral motions 17 a and 19 a of apparatus 10 a and the lateral motions17 b and 19 b of apparatus 10 b are independent so that the two devicescan be moved for pan and vergence. However, vertical motions 16 a and 18a of apparatus 10 a and 16 b and 18 b of apparatus 10 b are coupledtogether either mechanically or through computer control for tiltcontrol, as will be appreciated by those skilled in the art. Likewise,the rectangular displacements 20 a and 20 b are synchronized forsuper-resolution integration. These independent and synchronizeddisplacements can be easily accomplished by computer control 21 a and 21b (each similar to computer control 21 of apparatus 10 of FIG. 1 andmutually communicating by a communication link 23) or by a unitarycomputer control 24.

[0037] In FIG. 3, a greatly enlarged field of coverage 10 of a lens 12is illustrated. Each rectangle 26 represents a potential location of theimager 10 in the lens' field of coverage as intended for capture by 35mm film. “Additional areas” 28 are not typically used in filmphotography, but are accessible to the present invention. In fact, theentire area 12 can be sampled by the method and apparatus of thisinvention. For example, a ⅓ inch 4:3 imager provides 5 times lateral and5 times vertical coverage, for an effective field of 3200 by 2400 (ormore) for a nominal 640 by 480 pixel imager 10, selected as the“standard” image size under NTSC standards.

[0038] As noted previously, in the present invention, an imaging deviceis placed on a three-degree-of-freedom linear motion mechanism whoseposition can be controlled by a computer. Relatively broad fieldcoverage is preferably attained through use of a wide-anglelarge-coverage (“wide,” e.g. 45 degree or larger field of view) lens.High resolution is attained by placing the imaging device under the widelens, so the sensor's immediate field of view is considerably smallerthan that provided by the lens.

[0039] For example, for a 6 mm-wide imaging device under a 50 mm lens,the lens' 45 degree field of view is reduced to approximately45×(6/35)=7.7 degrees at the sensor array, making it act as anapproximately 400 mm lens at the sensor, and increasing pixel resolutionby a factor of about 6 in each direction. Super-resolution integrationtechniques can more than double this to a factor of about 12 in eachdirection, as will be appreciated by those skilled in the art.

[0040] Computer control of the motion platform upon which the sensorarray resides enables positioning accuracy in the micron range. Steppermotors and other precision actuators for accomplishing this control arewell known to those skilled in the art. Stepper motors operating withfull step positioning can position a platform at discrete positionsalong an axis to within a few microns. Differential stepper controlenables increasing the accuracy of this placement by one to two ordersof magnitude.

[0041] For fine determination of absolute imager location, a pair ofinterferometer or sonar-based ranging systems can be provided thatmeasure the positions of the sensors in the focal plane to sub-micronprecision during image capture. Since each imaging cell is about 10microns across in a current embodiment, this enables sufficientsub-pixel precision to keep the range estimates coherent across theoperational field of view. In the present example, interferometricmeasurement and differential stepper control may provide better than 3bits of sub-pixel localization over the range of motion

[0042] With reference to FIGS. 4 and 5, a method is described forutilizing a multi-camera in accordance with the present invention inorder to adjust the range of available stereoscopic imaging. FIG. 4illustrates a multi-camera 22 including a pair of monocular apparatus 10a and 10 b. The corresponding elements of apparatus 10 a and 10 b aredescribed above with reference to FIGS. 1 and 2. In FIG. 4, the imager14 a is centered about a focal axis of the lens 12 a and the imager 14 bis centered about a focal axis of the lens 12 b. Given this arrangement,the bounds (at least along the plane of the paper) of the view selectionof the imager 14 a are indicated by the lines 110 and 112, and thebounds (at least along the plane of the paper) of the view selection ofthe imager 14 b are indicated by the lines 114 and 116. The hatchedregion 118 indicates range of available stereoscopic imaging availablefor the arrangement of FIG. 4.

[0043] In FIG. 5, the imager 14 b has been repositioned along the X axisand, as a result, the range of available stereoscopic imaging availablefor the arrangement of FIG. 5 includes not only the hatched region 118,but now includes the hatched region 120. By making similar adjustmentsalong the Y axis, the range of available stereoscopic imaging can bevaried as desired.

[0044] An important design consideration for the present invention isthe realization that the alignment of multiple imagers for rangecomputation is important (recall the discussion of parallel epipolargeometry in the background). However, it is contemplated that themultiple imagers may be built as independent components and arrayed asdesired for particular ranging situations. Accordingly, the presentinvention teaches an alignment mechanism operable to align the multipleimagers in order to substantially meet the desired parallel epipolargeometry. One preferred alignment mechanism that will enable rapidalignment includes a laser diode system directed parallel to the imagingsurface of and exiting a first imager housing the laser diode. When theimagers are properly aligned, the beam enters another imager, isreflected on to the next imager, and so on, such that the beam returnsin a single plane of travel to the originating source. In this way,visual alignment of multiple imagers can be obtained to thesub-millimeter level with simple integrating signal processing andoptimization at the detector elements, as will be appreciated by thoseskilled in the art.

[0045] With reference to FIGS. 6 and 7 (top view and side view,respectively), an alignment mechanism 150 for aligning two movingimagers in accordance with one embodiment of the present invention willnow be described. The alignment mechanism 150 is jointly housed withinmoving imagers 152 and 154 and includes a laser diode 156, mirrors 158and 160, and a photosensor 162. The laser diode 156, the mirrors 158 and160, and the photosensor 162 are arranged such that, when the movingimagers 152 and 154 are properly aligned, a beam of light 164 generatedat the laser diode 156 is reflected within the moving imager 152 anddirected back to the photosensor 162. Thus proper alignment can beachieved by adjusting the moving imagers 152 and 154 until the lightbeam 164 is detected at the photosensor 162. Mechanical positionadjusters, such as those available from Newport Inc. of Irvine, Calif.,may be used to effect the imager alignment.

[0046] Note that the alignment mechanism 150 locates the laser diode156, mirrors 158 and 160, and the photosensor 162 within the imagers, asopposed to locating them externally. Doing so enables better alignmentthrough properly machining the light paths internal to the imagers. Aswill further be appreciated, the strategy of FIGS. 6 and 7 can beadapted to align a multiplicity of imagers using a single laser diodeand additional mirrors.

[0047] Another design consideration for the present invention is therealization that the precise measurement of location of the sensor ismore important than positioning accuracy. For most view selections,there is no need to achieve a particular exact direction of view: anapproximate direction will be sufficient. In other words, the sensordoes not need to be at a specific position under the lens and yet,wherever the sensor is placed, knowledge of its position is preferred toa quite high precision (e.g. a tenth of a micron). As an illustrativeexample, the sensor array could be located at an arbitrary position withrespect to the imaging plane, as long as the scene detail of interest isimaged and the system measures the imager's actual location with a highdegree of precision. The precision of this placement determines theaccuracy of the depth computed.

[0048] Yet another design consideration for the present invention is therealization that super-resolution computation is facilitated by havinghigh positioning accuracy over a distance of about ½ pixel (see Tagamiet al.'s U.S. Pat. No. 5,402,171). The alternative is quite acceptable(i.e. knowing to high precision where the other images contributing tothe integration are located). But, knowing the displacements in advanceand then using the known displacements during processing (for example,exactly one half pixel displacement in each direction) is a preferablesolution.

[0049] There are a number of enhancements to the basic methods andapparatus described above with reference to FIGS. 1-7. It should benoted that the present invention includes an instantaneous wide-angleviewing capability. Due to this wide-angle viewing capability, therequirement of moving the imager over the whole field of view whileintegrating to obtain a full view is eliminated. Instead, preferablythrough use of a small positive-curvature converter lens between thelens and the imager, the full field of view is simultaneously projectedonto the centrally-located sensor.

[0050] Another enhancement to the invention is the inclusion of acomputer-controlled lens focusing mechanism, in addition to theback-plane focusing means provided by the platform out-of-plane motion,and this would preferably use stereo-computed range to determinefocusing distance. Another enhancement is a computer-controlled zoommechanism that will allow adaptive selection of effective field of viewover a wide range of values. The implementation of computer-controlledlens focusing and zoom mechanisms are well known to those skilled in theart.

[0051] Thus one particular embodiment of the present invention includesa three-dimensional-adjusting imaging camera that has three motionplatforms coupled for three-dimensional movement, a controller formoving the platforms separately or in concert, a measurement system fordetermining the location of the platforms in an external frame ofreference, an imaging device positioned for three-dimensionaldisplacements on the coupled platform, and a lens whose focal surfacecoincides with the imaging surface of the imaging device underthree-dimensional movement of the motion platform and which has a fieldof coverage considerably larger than the optically sensitive area of theimaging device.

[0052] A three-dimensional-adjusting imaging camera 200 in accordancewith such an embodiment will now be described with reference to FIGS. 8and 9. The imaging camera 200 includes a pair of positionable imagingapparatus 202 a and 202 b that are operated in concert. The imagingapparatus 202 a includes motors 204 a, 206 a, and 208 a, linear slides210 a, 212 a, and 214 a, an imager 216 a, and three platforms 218 a, 220a, and 221 a.

[0053] The motor 208 a drives the platform 220 a along the linear slide214 a, the motor 204 a drives the platform 218 a along the linear slide212 a, and the motor 206 a drives the platform 221 a along the linearslide 210 a. The imager 216 a is mounted upon a surface of the platform221 a. Motion along the linear slide 214 a is perpendicular to motionalong the linear slide 212 a, but motion along both linear slides 212 aand 214 a is in-plane motion. Motion along the linear slide 210 a isout-of-plane motion. Hence, in-plane positioning is effectuated byactuating the motors 204 a and 208 a, while out-of-plane motion iseffectuated by actuating the motor 206 a. Operation of the positionableimaging apparatus 202 b is similar and should be self-evident.

[0054] The camera can be used for the automated tracking of objects inthe field of view and the automated scanning of a field of view. Inaddition, the use of such a camera for integrating images from differentpositions can be used to develop higher resolution composite images ofthe field of view, and for the automated range computation to featuresand elements in the field of view. A lower-resolution version of thelatter is preferably accomplished with a converting lens positionedbetween a primary lens and imaging device surface such that a largeportion of the field of view of the lens is projected onto the opticallysensitive area of the centrally-located imager.

[0055] As noted, the present invention may further include a computercontrolled focusing mechanism aside from the back-plane focusing means.In addition, the camera of the present invention may further include acomputer-controlled focal-length adjusting mechanism.

[0056] A further advantage accruing from moving the imagers undercomputational control is that shuttering and acquisition can besynchronized to ensure that images are free of motion blur—either fromcamera motion or from observed object motion.

[0057] The present invention has a great number of valuable applicationsin that any application of computational video processing, includingobservation, tracking, and stereo ranging, would benefit from manyelements of the present camera design. These benefits include simplifiedstereo matching, the retention of a single frame of reference, highresolution and wide field of view with inexpensive off-the-shelf lenses,field of view selection without camera or lens movement, andsuper-resolution.

[0058] An alternative to the above-described imaging devices is animaging device which utilizes a spherical lens. The human eye is theprincipal capture-device model for theoretical aspects of computervision research. Unfortunately, imager fabrication constraints haveprevented development of imaging devices that can exploit the geometryof this natural “device.” Previously, researchers have typically onlyhad access to planar focal-surface sensors such as those described abovewith reference to FIGS. 1-9.

[0059] In FIG. 10, a camera 30 includes a spherical lens 31 centered onan arbitrary spherical coordinate system and having a lens focal sphere32. The camera 30 further includes an elevation control axis 34 and anazimuth control axis 36. These axes may be implemented with curvedcontrol paths. A positioner 38 (shown here broken away for clarity) isattached to the axes 34 and 36 and is used to hold an imager 40 at theintersection of the axes. The actual position of positioner 38 is shownat 38′. A spherical-faced fiber-optic faceplate 42 is attached over theimager 40 to define the spherical image formation surface of the camera.An elevation controller 44 is coupled to the elevation control axis 34,and an azimuth controller 46 is coupled to the azimuth control axis 36.The elevation controller 44 and azimuth controller 46 are preferablycomputer controlled by a computer controller 48.

[0060] A curved focal surface, such as the retina and the sphericalsurface of the faceplate 42 described above, has a number of advantagesfor computational image processing, including: 1) excellent opticalresolution attainable when the focal surface needn't be made planar; 2)the simplicity of rotational over linear displacements; 3) a greatlyincreased undistorted field of view; 4) exponentially less lensilluminance fall-off with eccentricity; 5) greater effective pan andtilt velocities; and 6) opportunity to study (via simulation) aspects ofhuman psychophysical behavior.

[0061] With the present invention, the capture of spherical focal-planeimagery through, for example, the use of the spherical-faced fiber-opticfaceplate 42 provides these and other benefits. The present inventionprovides a mechanism for exploiting this in a spherical-imaging camerathat shares many of the design considerations of the above-describedlinear moving-imager camera. The present design uses a high resolutionimaging device (1K by 1K elements with 12 bits of resolution at eachpixel), mounted behind a fiber-optic faceplate that transfers thefocal-plane light pattern to the sensor elements.

[0062] Rotational mechanisms allow azimuth and elevation controls on theimager-faceplate's position on a sphere, with mechanisms similar tothose used in our linear-motion device described above for measuringthis position to sub-pixel precision.

[0063] A suitable lens for this embodiment of the present invention is aball lens, which is described in J. Baker's article entitled“Spherically Symmetrical Lenses and Associated Equipment for Wide AngleAerial Photography,” found in § 16.1 of the Nov. 30, 1949, REPORT OFUNITED STATES OFFICE OF SCIENTIFIC RESEARCH AND DEVELOPMENT, NATIONALDEFENSE RESEARCH COMMITTEE, which is incorporated herein by reference.Attachment of a fiber-optic faceplate to such a lens for astronomicstudy is described in Park et al.'s article entitled “Realtime TrackingSystem for the Wide-Field-of-View Telescope Project,” found in SPIE VOL.1111, ACQUISITION, TRACKING, AND POINTING III, 196-203 (1989) and Lewiset al.'s article entitled “WFOV Star Tracker Camera,” found in SPIE VOL1478, SENSORS AND SENSOR SYSTEMS FOR GUIDANCE AND NAVIGATION, 2-12(1991), both of which are incorporated herein by reference.

[0064] A lens similar in design character, although with a narrowerfield of view, is manufactured by the Optical Corporation of Americaunder the name “Wide Field-of-View Star Tracker.” The curvature of thefaceplate is selected by its radius from the center of rotation. Mappingof the resulting image geometry for binocular and motion disparitycomputation is straightforward, as will be appreciated by those skilledin the art. A smaller, lower resolution, and mechanically simplerversion of this stereo eyeball camera may be attained by placement of asingle imaging surface with fiber-optic faceplate behind each lens ofthe two-lens camera. As those skilled in the art will appreciate,rotation of the lenses and imagers in this eyeball system will notaffect the frame of reference, as this imager's lenses have no singleoptical axis, and the interocular baseline remains fixed.

[0065] As well as being used for range computation, thespherical-imaging embodiment of the present invention can beadvantageously used to study the human eye, examining elements of itsmotion, stereo, and even monocular processing: tracking, saccades, microsaccades, foveation, and other such aspects.

[0066] Another aspect of the present invention teaches a method forsynchronization of the moving imager with image capture in order todecrease blur resulting from the imager's movement.

[0067] The concept of a moving image-capture device presents a problemin that moving while capturing induces blur. If capture of blurredimages is to be avoided, control of the moving imager's motions must besynchronized with the image capture process. If this synchronization canbe attained, it presents the possibility of minimizing blur in twoforms. First, if all platform motion can be performed in the intervalbetween image capture and the start of the next integration period, thenno platform-induced motion blur will be observed—the imagers will bestationary while integrating. Second, if a moving object is to beimaged, then movements of the platform tailored to the motion of theobject will enable higher resolution—minimal blur—imaging of it andanything else moving with its velocity. The preferred embodiment of themoving-imager camera synchronizes image integration and capture tominimize these capture artifacts.

[0068] As will be appreciated by those skilled in the art, video imagersgenerally operate at fixed scan frequencies. A usual mode of captureinvolves having the imager acquire light energy for a period of time(the integration time), and then to stop the integration process andtransmit out the acquired information. The time between the stop ofcapture (the end of the integration) and the start of the nextintegration period presents an opportunity for platform motion. Thistime may be from six to tens of milliseconds, depending on the length ofthe next integration period. In certain embodiments of this invention,these timing signals are available to the moving imager controller, andplatform motion is effected in the period between the termination of oneintegration period and the beginning of the next.

[0069] In another embodiment of this invention, the control signals tobegin and end image integration are controlled by the moving imagercomputer such that integration is begun only after the intended motionis complete. Note that in the latter embodiment a fixed or standardimage frame rate may not be attained but, for computational purposes asopposed to videography purposes, fixed or standard frame rates are notessential. Here, the quality and character of the acquired imagery arethe important factors and, where a clear image is desired, the controlmethod described will enable this capture.

[0070] The information used to control use and generation of thesesynchronization timing signals may come from measurement devices such asthe interferometer or sonar-based systems described earlier, or fromprecalibration experiments in which the velocity and timingcharacteristics of the platforms are measured and tabulated forsubsequent use.

[0071] While this invention has been described in terms of severalpreferred embodiments, there are alterations, permutations, andequivalents which fall within the scope of this invention. As will beappreciated by the above descriptions and the drawings, the imagercontrol methods and apparatus of the present invention provide a stableframe of reference for monocular and multiple image computer analysisfor high-precision tracking, scene scanning, and range computation overa wide field of view and with detail of interest at a wide variety ofdistances. However, it should be noted that those skilled in the artwill realize that there are alterations, permutations, and equivalentsas fall within the true spirit and scope of the present invention.

1. A moving imager camera comprising: a first positioning mechanism capable of three-dimensional movement; a first imaging device having an imaging surface, the first imaging device being mounted upon a surface of the first positioning mechanism such that the first imaging device moves in concert with motion of the surface of the first positioning mechanism; a first measurement system operable to determine a position of the first imaging device within an external frame of reference defined by three axes X, Y, and Z; and a first lens having a focal surface and a field of coverage, the first lens arranged such that the focal surface coincides with the imaging surface of the first imaging device, the field of coverage of the first lens being larger than the optically sensitive area of the imaging surface.
 2. A moving imager camera as recited in claim 1 further including a computer system for controlling motion of the first positioning mechanism.
 3. A moving imager camera as recited in claim 1 wherein the first positioning mechanism includes first and second platforms, the first platform operable to move along the X axis of the external frame of reference, the second platform operable to move along the Y axis, the surface of the first positioning mechanism being arranged such that the surface moves in concert with the first and second platforms.
 4. A moving imager camera as recited in claim 3 wherein the first positioning mechanism further includes a third platform operable to move along the Z axis of the external frame of reference and the surface of the first positioning mechanism is an upper surface of the third platform.
 5. A moving imager camera as recited in claim 3 wherein the first positioning mechanism further includes at least one actuator for actuating one of the first and second platforms.
 6. A moving imager camera as recited in claim 5 further including a control system for controlling the operation of the at least one actuator.
 7. A moving imager camera as recited in claim 6 wherein the at least one actuator is a stepper motor.
 8. A moving imager camera as recited in claim 7 wherein the control system is a differential control system capable of microstepping the stepper motor.
 9. A moving imager camera as recited in claim 6 wherein the at least one actuator is a servo motor.
 10. A moving imager camera as recited in claim 1 wherein the imaging device is a digital imaging device.
 11. A moving imager camera as recited in claim 11 wherein the digital imaging device includes a photosensitive integrated circuit, the photosensitive integrated circuit having an array of photosensitive elements that is the imaging surface of the imaging device.
 12. A moving imager camera as recited in claim 11 wherein the photosensitive integrated circuit is operable to process signals generated at the array of photosensitive elements.
 13. A moving imager camera as recited in claim 10 wherein the imaging surface of the first imaging device is an array of imaging pixels, the resolution of the array being at least 512 pixels by 512 pixels.
 14. A moving imager camera as recited in claim 1 wherein the first measurement system includes an interferometer system operable to determine the position of the first imaging device in the focal plane.
 15. A moving imager camera as recited in claim 1 wherein the first measurement system is a sonar based ranging system operable to determine the position of the first imaging device in the focal plane.
 16. A moving imager camera as recited in claim 1 wherein the first lens is a lens designed for a 35 millimeter camera.
 17. A moving imager camera as recited in claim 1 wherein the first lens is a lens designed for an advanced photo system (APS) camera.
 18. A moving imager camera as recited in claim 1 further comprising a computer controlled focusing mechanism.
 19. A moving imager camera as recited in claim 18 wherein the computer controlled focusing mechanism is a lens focusing mechanism.
 20. A moving imager camera as recited in claim 18 wherein the computer controlled focusing mechanism is a back focusing mechanism.
 21. A moving imager camera as recited in claim 18 wherein the computer controlled focusing mechanism uses stereo-computed range to determine focusing distance.
 22. A moving imager camera as recited in claim 1 further comprising a computer controlled zoom mechanism providing adaptive selection of the effective field of view.
 23. A moving imager camera as recited in claim 1 further comprising a second positioning mechanism capable of three-dimensional movement; a second imaging device having an imaging surface, the second imaging device being mounted upon a surface of the second positioning mechanism such that the second imaging device moves in concert with motion of the surface of the second positioning mechanism; a second measurement system operable to determine a position of the second imaging device in the focal plane; and a second lens having a focal surface and a field of coverage, the second lens arranged such that the focal surface of the second lens coincides with the imaging surface of the second imaging device, the field of coverage of the second lens being larger than the optically sensitive area of the imaging surface.
 24. A moving imager camera as recited in claim 23 further including a control system operable to control motion of the first and second positioning mechanisms.
 25. A moving imager camera as recited in claim 24 wherein the control system includes a first controller arranged to control the first positioning mechanism and a second controller arranged to control the second positioning mechanism.
 26. A moving imager camera as recited in claim 23 further comprising an alignment mechanism operable to align the first and second imaging devices into substantially the same plane.
 27. A moving imager camera as recited in claim 23 wherein the alignment mechanism includes a laser diode disposed within the first positioning mechanism and aimed at a mirror disposed within the second positioning mechanism, and a light sensor disposed within the first positioning mechanism, such that when the first and second positioning mechanisms are aligned into substantially the same plane, a beam of light generated by the laser diode will reflect off the mirror and return for measurement and optimization at the light sensor.
 28. A moving imager camera as recited in claim 23 wherein the control system is operable to position the first and second positioning mechanisms in order to adjust a range of stereoscopic imaging available through the use of the first and second imaging devices.
 29. A moving imager camera as recited in claim 1 wherein the first positioning mechanism, the first imaging device, the first measurement system, and the first lens all comprise a first moving imager, the first moving imager being one of a plurality of moving imagers.
 30. A moving imager camera as recited in claim 29 further including an alignment mechanism operable to align the plurality of moving imagers into substantially the same plane.
 31. A moving imager camera as recited in claim 1 further including a converting lens arranged between the first lens and the imaging surface of the first imaging device such that a large portion of the field of view of the first lens is projected onto the imaging surface.
 32. A moving imager camera operable to generate stereoscopic images suitable for use in range computation, the moving imager camera comprising: a first moving imager including: a first positioning mechanism capable of two-dimensional movement; a first imaging device having an imaging surface, the first imaging device being mounted upon a surface of the first positioning mechanism such that the first imaging device moves in concert with motion of the surface of the first positioning mechanism; a first measurement system operable to determine a position of the first imaging device in the focal plane within an external frame of reference; and a first lens having a focal surface and a field of coverage, the first lens arranged such that the focal surface coincides with the imaging surface of the first imaging device, the field of coverage of the first lens being larger than the optically sensitive area of the imaging surface; and a second moving imager including: a second Positioning mechanism capable of two-dimensional movement; a second imaging device having an imaging surface, the second imaging device being mounted upon a surface of the second positioning mechanism such that the second imaging device moves in concert with motion of the surface of the second positioning mechanism; a second measurement system operable to determine a position of the second imaging device in the focal plane within an external frame of reference; and a second lens having a focal surface and a field of coverage, the second lens arranged such that the focal surface coincides with the imaging surface of the second imaging device, the field of coverage of the second lens being larger than the optically sensitive area of the imaging surface; and a computer system operable to control the first and second moving imagers in order to adjust a range of stereoscopic imaging available through the use of the first and second moving imagers.
 33. A moving imager camera as recited in claim 32 wherein the first positioning mechanism includes a first platform operable to move along both X and Y axes, the surface of the first positioning mechanism being arranged such that the surface of the first positioning mechanism moves in concert with the first platform.
 34. A moving imager camera as recited in claim 33 wherein the first positioning mechanism further includes at least one stepper motor for actuating the first platform.
 35. A moving imager camera as recited in claim 34 further including a control system for controlling the operation of the at least one stepper motor.
 36. A moving imager camera as recited in claim 35 wherein the control system is a differential control system capable of microstepping the at least one stepper motor.
 37. A moving imager camera as recited in claim 32 where the first imager is a digital imager, the imaging surface of the first imaging device is an array of imaging pixels, and the resolution of the array is at least 512 pixels by 512 pixels.
 38. A moving imager camera as recited in claim 32 wherein the first measurement system includes an interferometer system operable to determine the position of the first imaging device in the focal plane.
 39. A moving imager camera as recited in claim 32 wherein the first measurement system is a sonar based ranging system operable to determine the position of the first imaging device in the focal plane.
 40. A moving imager camera as recited in claim 32 wherein both the first and second lenses are designed for use in a 35 millimeter camera.
 41. A moving imager camera as recited in claim 32 further comprising an alignment mechanism operable to align the first and second imagers into substantially the same plane.
 42. A moving imager camera as recited in claim 41 wherein the alignment mechanism includes a laser diode disposed within the first imager and aimed at mirrors disposed within the second imager, and a light sensor disposed within the first imager, such that when the first and second imagers are aligned into substantially the same plane, a beam of light generated by the laser diode will reflect off the mirrors and return for measurement at the light sensor.
 43. A moving imager camera comprising: a first positioning mechanism capable of motion along two degrees of freedom; a first imaging device having an imaging surface, the first imaging device being mounted upon a surface of the first positioning mechanism such that the first imaging device moves in concert with motion of the surface of the first positioning mechanism; a first measurement system operable to determine a position of the first imaging device within an external frame of reference defined by azimuth and elevation axes; and a first spherical lens having a spherical focal surface and a field of coverage, the first spherical lens arranged such that the spherical focal surface coincides with the imaging surface of the first imaging device, the field of coverage of the first lens being larger than the optically sensitive area of the imaging surface.
 44. A moving imager camera as recited in claim 43 wherein the first positioning mechanism operates such that the two degrees of freedom correspond to motion along the azimuth and elevation axes respectively.
 45. A moving imager camera as recited in claim 44 wherein the first positioning mechanism includes a first curved control path substantially aligned with the azimuth axis, and the first imaging device is coupled to the first positioning mechanism such that the first imaging device can move along the first curved control path.
 46. A moving imager camera as recited in claim 44 wherein the first positioning mechanism includes a second curved control path substantially aligned with the elevation axis, and the first imaging device is coupled to the first positioning mechanism such that the first imaging device can move along the second curved control path.
 47. A moving imager camera as recited in claim 43 wherein the imaging surface is formed to substantially correspond to the spherical focal surface of the first spherical lens.
 48. A moving imager camera as recited in claim 47 wherein the imaging surface includes an array of fiber optic strands, the exposed surface of each fiber optic strand formed such that the imaging surface substantially corresponds to the spherical focal surface of the first spherical lens.
 49. A moving imager camera as recited in claim 43 wherein the moving imager camera further comprises: a second positioning mechanism capable of motion along two degrees of freedom; a second imaging device having an imaging surface, the second imaging device being mounted upon a surface of the second positioning mechanism such that the first imaging device moves in concert with motion of the surface of the second positioning mechanism; and a second measurement system operable to determine a position of the second imaging device within the external frame of reference; and a second spherical lens having a spherical focal surface and a field of coverage, the second spherical lens arranged such that the spherical focal surface of the second lens coincides with the imaging surface of the second imaging device, the field of coverage of the second lens being larger than the optically sensitive area of the imaging surface of the second imaging device.
 50. A moving imager as recited in claim 49 further including a computer system operable to position the first and second imaging devices in order to obtain a larger field of stereoscopic images.
 51. A moving imager as recited in claim 43 wherein the first positioning mechanism, the first imaging device, the first measurement system and the first spherical lens together comprise a first moving imager, and the first moving imager is one of a plurality of moving imagers comprising the moving imager.
 52. An imager camera suitable for computational ranging, the imager camera comprising: a first imaging device having an imaging surface; a first spherical lens having a spherical focal surface and a field of coverage, the first spherical lens arranged such that the spherical focal surface coincides with the imaging surface of the first imaging device, the field of coverage of the first lens being larger than the optically sensitive area of the imaging surface; a second imaging device having an imaging surface; a second spherical lens having a spherical focal surface and a field of coverage, the second spherical lens arranged such that the spherical focal surface of the second spherical lens coincides with the imaging surface of the second imaging device, the field of coverage of the second lens being larger than the optically sensitive area of the imaging surface of the second imaging device.
 51. A method for reducing blur in a moving imager camera, the moving imager camera having a lens and a moving imager having an imaging surface residing in the focal plane of the lens, the method involving the steps of: receiving an indication that the moving imager, positioned at a first position during a first image capture step, must be at a second position during a second image capture step that is performed after the first image capture step; determining an inactive time period between the stop of the first image capture step and the start of the second image capture step; determining that the first image capture step has stopped; and moving the imager to a desired new position during the inactive time period, whereby the moving imager is properly positioned and substantially stable in the focal plane of the lens during the second capture step.
 52. A method as recited in claim 51 wherein the inactive period corresponds to a time period during which the imager is not acquiring light energy.
 53. A method for reducing blur in a moving imager camera, the moving imager camera having a lens and a moving imager, the moving imager having an imaging surface residing in the focal plane of the lens, the method involving the steps of: receiving an indication that an object that the moving imager camera is intended to capture will be moving; determining the impending motion of an image of the object that is projected onto the imaging surface; moving the imaging surface in accordance with the image of the object, whereby the moving imager captures the image of the object even though the object is moving.
 54. A moving imager camera comprising: a first positioning mechanism capable of two-dimensional movement; a first imaging device having an imaging surface, the first imaging device being mounted upon a surface of the first positioning mechanism such that the first imaging device moves in concert with motion of the surface of the first positioning mechanism; a first measurement system operable to determine a position of the first imaging device within an external frame of reference; a first lens having a focal surface and a field of coverage, the first lens arranged such that the focal surface coincides with the imaging surface of the first imaging device, the field of coverage of the first lens being larger than the optically sensitive area of the imaging surface; and a converting mechanism having a converting lens, the converting mechanism operable to dispose the converting lens in a first position between the first lens and the first imaging device, the converting lens operable such that when it is disposed in the first position a large portion of the field of view of the first lens is projected onto the imaging surface. 